Augmented Reality (AR) is the addition of digital imagery and other information to the real world by a computer system. AR enhances a user's view or perception of the world by adding computer generated information to their view. Spatial Augmented Reality is a branch of AR research that employs projected light to present perspective corrected computer graphics that directly illuminate physical objects to enhance their appearance. In addition to using multiple projectors to projects information directly onto objects of interest, SAR systems can also use tracking systems and sensors to build up a three dimensional (3D) model of the SAR environment, and can track movement of real world objects. Such movements or changes are integrated into the 3D model so that updates can be made to projections as objects are moved around.
One application of SAR is for developing and evaluating user interfaces for workstations, as well as the layout of workstations in a control room. In these applications a simple substrate, or physical prototype, can be constructed with the desired shape for each workstation, or piece of a workstation. SAR can be used to visualize different control features on each physical prototype and different physical placements of the workstations. Users can directly interact with the augmented substrate without having to wear or hold a display device, and the projected information can be viewed by multiple people at the same time. Multiple projectors are often used to reduce shadows while users interact with the workstation, or to provide complex or extended displays.
In such systems where multiple projectors are used, calibration must be performed to ensure alignment of projected images. A calibration process is required to calculate both the intrinsic parameters of the projector, such as the horizontal and vertical field of view, and the extrinsic parameters, such as the projector's position and orientation relative to the world. This is commonly accomplished by matching projector pixels with known 3D points in the world, such as features on a physical object.
These correspondences can be found manually with a projected crosshair visually displayed on the physical object using a keyboard or mouse for adjustment, as used with Shader Lamps. This process can be automated. One approach to projector calibration is to employ a calibrated camera. These projector-camera systems allow for real-time image adjustment that allows images to be displayed onto surfaces that are not traditionally designed for projections, such as non-planar geometry and textured surfaces.
A second method of projector calibration employs photo detectors to find projector alignment. Several research projects have proposed projector based position detection employing photo detectors in a projection volume. These photo detector based measurements allow a direct mapping between the real environments and projection information without consideration of a camera coordinate system. These have included structured light approaches known as Gray-coding which uses a sequence of images to find the pixel locations of the photo detectors, on the basis that a portion of a display surface illuminated by multiple projectors will be brighter. Once the projector-world correspondences are found, the calibration parameters for the projector can be calculated. An algorithm for calculating these parameters is described in detail by O. Bimber and R. Raskar, Spatial Augmented Reality: Merging Real and Virtual Worlds. A K Peters, Wellesley, 2005[1].
However, despite the extensive research, there are still a number of limitations that prevent the flexibility and performance required for multi-projector SAR systems. Current point based photo detectors can only locate positions with pixel accuracy, and thus images can appear ghosted and mis-aligned.
There is thus a need to provide improved methods and apparatus for estimating locations with sub-pixel accuracy to allow calibration and alignment of multiple projectors in SAR systems, or to at least provide users with a useful alternative.